Two Entropy-Based Methods for Learning Unsupervised Gaussian Mixture Models
نویسندگان
چکیده
In this paper we address the problem of estimating the parameters of a Gaussian mixture model. Although the EM (ExpectationMaximization) algorithm yields the maximum-likelihood solution it requires a careful initialization of the parameters and the optimal number of kernels in the mixture may be unknown beforehand. We propose a criterion based on the entropy of the pdf (probability density function) associated to each kernel to measure the quality of a given mixture model. Two different methods for estimating Shannon entropy are proposed and a modification of the classical EM algorithm to find the optimal number of kernels in the mixture is presented. We test our algorithm in probability density estimation, pattern recognition and color image segmentation.
منابع مشابه
Comparison between supervised and unsupervised learning of probabilistic linear discriminant analysis mixture models for speaker verification
We present a comparison of speaker verification systems based on unsupervised and supervised mixtures of probabilistic linear discriminant analysis (PLDA) models. This paper explores current applicability of unsupervised mixtures of PLDA models with Gaussian priors in a total variability space for speaker verification. Moreover, we analyze the experimental conditions under which this applicatio...
متن کاملRecognizing the Emotional State Changes in Human Utterance by a Learning Statistical Method based on Gaussian Mixture Model
Speech is one of the most opulent and instant methods to express emotional characteristics of human beings, which conveys the cognitive and semantic concepts among humans. In this study, a statistical-based method for emotional recognition of speech signals is proposed, and a learning approach is introduced, which is based on the statistical model to classify internal feelings of the utterance....
متن کاملImage segmentation based on maximum-likelihood estimation and optimum entropy-distribution (MLE-OED)
A novel method based on MLE–OED is proposed for unsupervised image segmentation of multiple objects with fuzzy edges. It adjusts the parameters of a mixture of Gaussian distributions via minimizing a new loss function. The loss function consists of two terms: a local content fitting term, which optimizes the entropy distribution, and a global statistical fitting term, which maximizes the likeli...
متن کاملModeling text with generalizable Gaussian mixtures
We apply and discuss generalizable Gaussian mixture (GGM) models for textmining. The model automatically adapts model complexity for a given text representation. We show that the generalizability of these models depends on the dimensionality of the representation and the sample size. We discuss the relation between supervised and unsupervised learning in text data. Finally, we implement a novel...
متن کاملFactoring Variations in Natural Images with Deep Gaussian Mixture Models
Generative models can be seen as the swiss army knives of machine learning, as many problems can be written probabilistically in terms of the distribution of the data, including prediction, reconstruction, imputation and simulation. One of the most promising directions for unsupervised learning may lie in Deep Learning methods, given their success in supervised learning. However, one of the cur...
متن کامل